Systems and methods for recovering information from nand gates array memory systems

ABSTRACT

A method is disclosed for recovering data associated with a damaged file stored in a NAND gate array memory. The method includes the steps of: identifying all meta data associated with the damaged file; identifying each logical block address of all identified meta data; collecting all physical block addresses associated with one of the identified logical block addresses or the identified meta data; counting in a replace table (ReplTable) a number of matches to a physical block address of the damaged file for each physical block address of the damaged file; choosing a block in a linked list that corresponds to the physical block address of the block in the linked list; and linking all chosen blocks to form a replicated file.

PRIORITY

The present application claims priority to U.S. Provisional PatentApplication Ser. No. 61/053,282 filed May 15, 2009, the entiredisclosure of which is hereby incorporated by reference.

GOVERNMENT SUPPORT

The invention was made with government support under Grants Nos.CCR-0073377, CCR-0312613 and SGER 0610538, each awarded by the NationalScience Foundation. The United States government has certain rights tothis invention.

BACKGROUND

The invention generally relates to data recovery systems, and relates inparticular to data recovery systems for NAND gate array memories.

NAND gate array memories are a popular technology used in flash memoriesdue, in part, to their low cost and high density (up to 16 Gb per chip).The technology has been used widely in handheld devices such as USBdrives, cell phones, touch phones, iPod™ devices and iPhones™ devicessold by Apple, Inc. of Cupertino, Calif., etc. Further future computingdevices such as laptops may also include such memory devices. Usersstore important information on these storage devices such as emails,photos, financial information, and personal data.

Device failures, however, such as hardware faults, OS failures, physicaldamages, virus attacks, and user errors sometimes occur resulting indata damages or data losses. Moreover, the charge stored in eachfloating gate of each transistor in a NAND gate array memory will leakin time, and blocks within each NAND gate array memory sometimes becomedamaged during use (e.g., during writing and erasing operations). Forthese reasons, NAND gate array memories are typically provided withspare memory area such that a bad block may be labeled as such, and datamay instead be diverted to good blocks within the NAND gate arraymemory. More importantly, each block in an NAND gate array flash haslimited number of erase cycles (10,000 for MLC and 100,000 for SLC). Aprocess called wear leveling is typically employed to spread the use ofdata blocks as uniformly as possible to provide that the blocks wear outmore evenly. When transistors and blocks become damaged, however, datatherein will be lost. There is a need, therefore, for techniques torecover data from such NAND gate array memories in case of failures.Furthermore, there is also a need in computer forensics to recover dataevidence from such handheld devices in case of physical damages or fileshaving been deleted.

Handheld devices are typically not as well protected as traditionalstorages such as disks, which may be maintained in air-conditioned datacenters and may be managed by information technology (IT) professionals.Mature data protection technologies such as traditional backup, snapshottechniques, continuous data protection (CDP), timely recovery to anypoint-in-time (TRAP) array, and Coupling Update by Parities (CUP) datatechnologies are available to protect data stored in hard disk drivesand data can be recovered in case of failures. Handheld devices are usedby the general public, and as a result, failures such as hardwarefaults, physical damages, and user errors occur more frequently thanwith hard disks that are maintained by IT professionals. In addition,majority of users do not do backup or snapshot for data stored on thesehandheld devices. Recovering data from such NAND gate array memoriespresents significant technical challenges.

U.S. Published Patent Application Publication No. 2008/0104308, forexample, discloses a technique to rebuild the block mapping table forthe purpose of providing a quick reboot from flash memory in the eventthat the mapping table is lost in the RAM memory. During rebooting, thelatest updated wear-sorted block list (WSBL) is read from non-volatilecache memory, and then the block mapping data is restored with referenceto erasing counts for the blocks. The system does not, however, providefor the recovery of data stored in a flash memory to a previous point intime.

U.S. Pat. No. 6,970,890 discloses a method of using base block copies todefine the location of selected data structures used for file systemmanagement. The method uses at least two of the base block copies toprovide redundancy so that in the event one of the copies cannot belocated or verified, the other copy can be used to rebuild the defectivebase block copy. This scheme, however, requires modification of filesystems to provide redundancy.

In an article titled Forensic Data Recovery From Flash Memory, by M.Breeuwsma, de Jongh, Martien, Klayer, Coert, van der Knijff, Ronald,Roeloffs, Mark, Small Scale Digital Device Forensics Journal, vol. 1,no. 1 (June 2007), it is disclosed that three low-level data acquisitionmethods may be employed for making full memory copies of flash memorydevices. Steps are also disclosed therein for translating the extracteddata into a format that may be understood by common forensic mediaanalysis tools. The systems of this article, however, do not make theraw data readable by upper layer file systems, and further are not ableto recover data to a previous point in time by tracing the block logsthat exist in NAND gate array memories and are invisible to file systemsor users.

There remains a need therefore, for a data recovery system for flashmemory storage that is able to recover data to a previous point in timein case of failures.

SUMMARY

In accordance with an embodiment, the invention provides a method forrecovering data associated with a damaged file stored in a NAND gatearray memory. The method includes the steps of: identifying all metadata associated with the damaged file; identifying each logical blockaddress of all identified meta data; collecting all physical bockaddresses associated with one of the identified logical block addressesor the identified meta data; counting in a replace table (ReplTable) anumber of matches to a physical block address of the damaged file foreach physical block address of the damaged file; choosing a block in alinked list that corresponds to the physical block address of the blockin the linked list; and linking all chosen blocks to form a replicatedfile. In accordance with another embodiment, the number of matches to aphysical block address of the damage file for each physical blockaddress of the damaged file is (MatchNo), and the step of choosing ablock in a linked list that corresponds to the physical block address ofthe block in the linked list involves choosing the (MatchNo+1)^(th)block in the linked list for each physical block address of the damagedfile.

In accordance with another embodiment, the invention provides a systemfor recovering data associated with a damaged file stored in a NAND gatearray memory. The system includes: meta data identification means foridentifying all meta data associated with the damaged file; logicalblock address means for locating each logical block address of allidentified meta data; collecting means for collecting all physical bockaddresses associated with one of the identified logical block addressesor the identified meta data; counting means for counting in a replacetable (ReplTable) a number of matches to a physical block address of thedamaged file for each physical block address of the damaged file;selection means for choosing a block in a linked list that correspondsto the physical block address of the block in the linked list; andlinking means for linking all chosen blocks to form a replicated file.

BRIEF DESCRIPTION OF THE ILLUSTRATED EMBODIMENTS

The following description may be further understood with reference tothe accompanying drawings in which:

FIG. 1 shows an illustrative diagrammatic view of a 2 Giga byte NANDgate array memory with which techniques of the invention may beemployed;

FIG. 2 shows an illustrative diagrammatic view of an internal structureof a NAND gate array block with which techniques of the invention may beemployed;

FIG. 3 shows an illustrative diagrammatic view of an address mappingtechnique for a NAND gate array memory in accordance with an embodimentof the invention;

FIG. 4 shows an illustrative diagrammatic view of a relationship betweena logical block address, a physical block address and a physical blockoffset in accordance with an embodiment of the invention;

FIG. 5 shows an illustrative diagrammatic view of an address mappingtechnique for a NAND gate array memory in accordance with an embodimentof the invention wherein a first file is created, then changed, thendeleted, and then a new file is created; and

FIG. 6 shows an illustrative diagrammatic view of procedural stepsemployed in a NAND gate array data recovery system in accordance with anembodiment of the invention.

The drawings are shown for illustrative purposes only.

DETAILED DESCRIPTION

A data recovery system and method are disclosed to recover NAND gatearray data to a previous point in time in case of failures. Based onobservations of physical properties of NAND gate arrays, the techniquetakes advantages of wear leveling and performance considerations. Thetechnique is able to recover data to a previous point in time in case ofdata loss and/or damage by reconstructing index structures and locatingcorresponding data blocks from a NAND gate array at the file systemlevel.

The technique is believed to work on all NAND gate array memories, andis able to recover data from NAND gate array memories in case of datadamages caused by hardware failures, user errors, operating systemcrashes, and virus attacks etc. The technique also works at both filesystem level and the physical layer level of flash memories.

An approach of the present invention is to make use of existing filesystem and wear leveling at the physical layer with no explicit dataredundancy provided at the file system level to recover data. Theapproach takes advantages physical level properties to recover data fromlogs of blocks to a previous point in time.

FIG. 1 shows the organization of a physical layer of a NAND gate array.As shown in FIG. 1, a physical layer of a 2 Giga byte NAND gate arraystorage 10 for use as flash memory includes a user data portion 12having, for example, 2,048 blocks per device, and each block 14 maycontain 64 pages. Each page contains 2 k bytes of memory, plus 64 bytesof spare memory. Each block, therefore, contains 128 k bytes of memory,plus 4 k bytes of spare memory. Blocks are the smallest erasable unitsand pages are the smallest programmable units. The NAND gate arraystorage 10 also includes a data register 16 and a cache register 18.Input and output operations (e.g., in 8 bit bytes) are executed throughthe cache register 18 as shown at 20, and the data register 16 maintainsthe routing and addressing of the data to good blocks in the user dataportion 12.

When a write operation is performed, the system first finds a free pageto which the data will be written. If there is no free page available,then an erase operation is necessary to create free pages. Readoperations usually takes about 25 microseconds whereas erase operationtakes 1.5 to 3 milliseconds. For performance considerations, thecontroller inside a flash memory always tries to delay executing eraseoperations as long as possible by searching for available free pages.

FIG. 2 shows the architecture of a block 22 in a NAND gate array memory.Each transistor 24 provides a memory cell and includes a control gate 26and a floating gate 28. When erased, each cell stores a high value ofone. A page is shown at 30 and a string is shown at 32.

Another important physical property of each block is that the lifetimeof the flash memory is limited by the number of erase operationsperformed on a block. Typically, a block can be erased only 10 k or 100k times in its lifetime. After that, the block becomes bad. For example,if a block were erased and reprogrammed every minute, every day forseven days (60×24×7=10,080), then the number of erase operations mayexceed the lifetime of that block—in just 7 days. As mentioned above, inorder to improve the lifetime of a flash memory, wear leveling istypically done by distributing erase operations more evenly across allblocks. For example, for the same flash memory with the life of 10 kerases, if erase operations are distribute to all 4 k blocks, then(10,000*4,096)/(24*60)=28,444 days, which translates to 77.9 years ofstorage life. Wear leveling is therefore an important feature that isimplemented in most NAND gate array memories.

When a file system is built on NAND gate array memories, a device driveris needed to provide block level services between a file system and theNAND gate array. This device driver (which includes the data register 16and cache register 18 of FIG. 1) is responsible for address mapping, rawdata block reading and writing, and wear leveling. There are two coredata structures involved in the implementation of a NAND device driver:an enumeration table (ENUTable) and a replace table (ReplTable). TheENUTable stores the mapping information from logical block address tophysical block address. The ReplTable makes a linked list to store thelog of blocks for each physically erasable block.

FIG. 3 shows the indexing and data structures of a typical NAND gatearray storage. A logic block address (LBA) 40 is addressed in theENUTable 42. An associated physical block address (PBA) 44 is then usedby the ReplTable 46 to access an available block 48. Each LBA (istherefore associated with a PBA 44 as well as a physical block offset(PBO) 50 as shown in FIG. 4.

The read and write operations of the NAND gate arrays are executed asfollows. When a read request is received, the device driver looks up theaddress in the ENUTable 42 and translates the LBA 40 address intophysical block address. By retrieving corresponding linked list for thePBA 44, the data block storing the newest data is selected and isreturned to the file system. When a write request is issued by the upperlayer file system, the device driver tries to find a free page in a freedata block to store the new data and inserts this data block into thecorresponding linked list of the physical block address. When storagespace is used up and a free data block cannot be found, the wearleveling algorithm will be performed. All data blocks in the longestlinked list will be erased after all data blocks in the linked list aremerged. During the merge process, only the freshest data is kept at onephysical data block. All physical data is, therefore, updated out ofplace, which is different from the update-in-place procedure used bytraditional hard disks.

The present invention leverages the indexing structure and the physicalplacement of data pages. The logs of blocks built in NAND gate arraystorage for the purpose of wear leveling and performance considerationsprovide us with opportunities to recover data to a previous good pointin time in case of hardware failures, virus attacks, and user error etc.When a file is changed and then deleted, information is recorded. Asshown in FIG. 5, for example, when File A is originally saved havingdata a, b, c (as shown at 60), metadata ma (as shown at 62) is created.The ReplTable 64 assigns blocks 66, 68, 70 for storing the data a, b,c., and assigns meta data ma to a meta data block 76. When File A islater changed to a′, b′, c′ (as shown at 72), metadata ma′ (as shown at74) is created, and the ReplTable 64 assigns new blocks 78, 80, 82 forstoring the data a′, b′, c′, and assigns meta data ma′ to a meta datablock 84. Now, when File A is deleted, and new File B having data d, e,f is created (as shown at 86), new meta data nib is created as shown at88. The ReplTable 64 then assigns blocks 90, 92, 94 for storing the datad, e, f, and assigns meta data nib to a meta data block 96. While allphysical data are updated out of place, the prior data from File A, theprior original data (a, b, c) or the changed data (a′, b′, c′) of File Amay be recovered.

The recovery algorithm works as follows using conventional computerprocessing hardware that accesses the NAND gate array memory via, forexample, a universal serial bus (USB) connection. With reference to FIG.6, the process of recovering a file X starts (step 100) by looking upthe data structures in a file system using, for example, a fileallocation table (FAT) to find all metadata information of the recoveredfile and corresponding LBAs (step 102). If a corresponding LBA is found(step 104), then the system finds the associated meta data in theReplTable, collects all PBAs of file X, searches all meta data inReplTable for matches to a PBA of X in other files, and counts thenumber of matches (MatchNo) for each PBA of X (step 106).

If no corresponding LBA is found (step 104), then the system goes toReplTable; traverses all linked lists of meta data and looks for a matchto X (step 108). If no match is found (step 110), then the systemprovides an indication that File X cannot be recovered (step 112). If amatch is found (step 110), then the system proceeds to step 106 andfinds the associated meta data in the ReplTable, collects all PBAs offile X, searches all meta data in ReplTable for matches to a PBA of X inother files, and counts the number of matches (MatchNo) for each PBA ofX.

Based on the meta data information, the ENUTable is used to locate thephysical locations of all metadata to the file, and all physicaladdresses therefore of the data blocks belonging to the File X arecollected (step 106). While looking for the metadata, file attributeinformation (such as times of creation and changes made to the file) isalso retrieved and analyzed. This information will be used inreconstruction of the file to be recovered. At the same time, A counter(MatchNo) is maintained for the number of appearances of the samephysical block address (PBA) in the metadata list.

This value MatchNo provides the number of overwrites to the data blockto be recovered by the file system. This value is used to pick up thedata block of the recovered file in the linked list by traversing thecorresponding linked list of the PBA and selected the (MatchNo+1)^(th)element in the linked list for recovery purpose (step 114). In this way,all data blocks of the file X to be recovered are collected. Thereafter,all the data blocks are collected, and the File X is reconstructed (step116). It is also possible to recover only a part of a file. If the logsof all data blocks have been erased, then the file may not be recovered.

With reference again to FIG. 5, wherein File A was first created andlater changed, then deleted and then partially overwritten with File B,because the file system considers File A having been deleted, it mayallocate the same LBAs for d and e of File B as a and b of File A,respectively. In the traditional storage, the write operations of d ande of file B would have overwritten a and b of File A. But, in the NANDgate array flash storage, b and c of File A are not overwritten butlinked in the linked list as shown at 68 and 70. Similarly, the newmetadata of file A, ma′, did not overwrite the old meta data of File A,ma. By tracing back the meta data and the linked blocks, we are able torecover File A as it was before the first changes were made, i.e., FileA consists of data a, b, and c, or as it was after the first changeswere made, i.e., File A consists of a′, b′, c′. During the filereconstruction process, file attribute information in metadata maybeused to facilitate the recovery process.

Hand held devices such as USB drive, PDA, cell phones, iPod, iPhones andTouchPhones etc use NAND gate array flash memory to store information.Some of the information is very important to users or to businesses. Itis important, therefore, to keep this important information safe,reliable and recoverable in case of failures. The techniques disclosedherein provide a method of recovering data from such NAND Gate arraymemories in case of data damage caused by hardware failures, usererrors, operating system crash, and virus attacks. The technique worksat both file system level and physical device level to recover deletedor damaged data in a flash storage. By leveraging the physicalproperties such as wear leveling and slow erases, data may be recoveredto a previous point in time when the data was not lost or corrupted.

Those skilled in the art will appreciate that numerous modifications andvariations may be made to the above disclosed embodiments withoutdeparting from the spirit and scope of the invention.

What is claimed is: 1.-17. (canceled)
 18. A method of recovering memorycell information associated with damaged blocks of a NAND gate arraymemory, said method comprising the steps of: identifying each logicalblock address associated with the memory cell information; collectingall physical block addresses associated with one of the identifiedlogical block addresses; counting in a replace table (ReplTable) anumber of matches to a physical block address of the damaged blocks foreach collected physical block address; choosing a corresponding block ina linked list that corresponds to the physical block address of thecorresponding block in the linked list; and linking all chosen blocks toprovide recovered memory cell information.
 19. The method as claimed inclaim 18, wherein said method collects all physical bock addressesassociated with the identified logical block addresses.
 20. The methodas claimed in claim 18, wherein said method further includes the step ofproviding an output of a recovered file with a time stamp.
 21. Themethod as claimed in claim 18, wherein said memory cell information isprovided by memory cells, each of which is a transistor.
 22. The methodas claimed in claim 18, wherein the recovered memory cell informationprovides at least a portion of a data file.
 23. A method of recoveringmemory cell information associated with damaged blocks of a NAND gatearray memory, said method comprising the steps of: identifying eachlogical block address associated with the memory cell information;collecting all physical block addresses associated with one of theidentified logical block addresses; counting in a replace table(ReplTable) a number (MatchNo) of matches to a physical block address ofthe damaged blocks for each collected physical block address, whereinsaid replace table assigns meta data to a meta data block; choosing acorresponding block in a linked list that corresponds to the physicalblock address of the corresponding block in the linked list by choosingthe (MatchNo+1)^(th) block in the linked list for each counted physicalblock address; and linking all chosen blocks to provide recovered memorycell information.
 24. The method as claimed in claim 23, wherein saidmethod collects all physical bock addresses associated with theidentified logical block addresses.
 25. The method as claimed in claim23, wherein said method further includes the step of providing an outputof a recovered file with a time stamp.
 26. The method as claimed inclaim 23, wherein said memory cell information is provided by memorycells, each of which is a transistor.
 27. The method as claimed in claim23, wherein the recovered memory cell information provides at least aportion of a data file.
 28. A system for recovering memory cellinformation data associated with a damaged block of a NAND gate arraymemory, said system comprising: logical block address means for locatingeach logical block address associated with the memory cell information;collecting means for collecting all physical block addresses associatedwith one of the identified logical block addresses; counting means forcounting in a replace table (ReplTable) a number of matches to aphysical block address of the damaged file for each physical blockaddress; selection means for choosing a corresponding block in a linkedlist that corresponds to the physical block address of the correspondingblock in the linked list; and linking means for linking all chosenblocks to provide recovered memory cell information.
 29. The system asclaimed in claim 28, wherein said collecting means collects all physicalbock addresses associated with the identified logical block addresses.30. The system as claimed in claim 28, wherein said system provides anoutput of a recovered file with a time stamp.
 31. The system as claimedin claim 28, wherein said memory cell information is provided by memorycells, each of which is a transistor.
 32. The system as claimed in claim28, wherein the recovered memory cell information provides at least aportion of a data file.